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Abstract 

Demixing is the problem of identifying multiple structured signals from a superimposed observation. This work analyzes 
a general framework, based on convex optimization, for solving demixing problems. We present a new solution to determine 
whether or not a specibc convex optimization problem built for generalized demixing is successful. This solution will also bring 
about the possibility to estimate the probability of success by the approximate kinematic formula. 

Index Terms 

Compressive sensing, £i-minimization. Sparse signal recovery. Convex optimization. Conic geometry. 


I. Introduction 


A ccording to the theory of convex analysis, convex cones have been exploited to express the optimality conditions for 
a convex optimization problem Q. In particular, Amelunxen et al. Q present the necessary and sufficient conditions 
for the problems of basis pursuit (BP) and demixing to be successful. 

Let xq C R" be an unknown fc-sparse vector with k nonzero entries in certain domain, let A be an m x n random matrix 
whose entries are independent standard normal variables, and let z = Axq G K™ be the measurement vector obtained via 
random transformation by A. In regard to the basis pursuit (BP) problem, which is defined as; 


(BP) minimize ||a;||i subject to z = Ax, (I.l) 

a convex optimization method was proposed by Chen et al. Q to solve the sparse signal recovery problem in the context of 
compressive sensing 0 when m < n. 

The authors are with Institute of Information Science, Academia Sinica, Taipei, Taiwan. E-mail: r00221004@ntu.edu.tw, spybeiman@gmail.com, and 


lcs@iis.sinica.edu.tw. Con'esponding author: C.-S. Lu. 
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To explore whether BP has a unique optimal solution, Amelunxen et al. Q start from the concept of conic integral. 
Definition I.l. (descent cone). The descent cone x) of a proper convex function f : K" —> K af the point x G K” is 


the conical hull of the perturbations that do not increase f near x. 

x) := y {y e R" : f{x + ry) < f{x)}. 


( 1 . 2 ) 


r>0 


We say that problem BP defined in Eq. <0 succeeds when it has a unique minimizer x that coincides with the true unknown, 
that is, X = Xq. To characterize when the BP problem succeeds, Amelunxen et al. present the primal optimality condition as: 


null(A) n ,a:o) = {0} 


(1.3) 


in terms of the descent cone 0 (cf., 0 and 0), where null{A) denotes null space of A. The optimality condition for the 
BP problem is also illustrated in Fig. 0 



Fig. 1. The optimality condition for the BP problem. [Left] BP succeeds. [Right] BP fails. S(xo) = {?; e S'* ; ||a:o + 3 /||i < ||a:o||i}. 


Amelunxen et al. 0 also explore the demixing problem (sparse + sparse) characterized as 

z = Xo + Uyo, ( 1 . 4 ) 

where U G is a known orthogonal matrix, and Xq is itself sparse and yg is sparse with respect to U. The optimization 

problem of recovering signals Xg and yg is formally defined as follow, which we call demixing problem (DP) in short: 

{ minimize ||a:||i 

(1.5) 

subject to ||y||i < ||yolii and z = a; + Uy. 

They propose the primal optimality condition (also illustrated in Fig. 0 as: 

^(||.||i,a:o)n-C/^(||.||,,yo) = {0} (1.6) 

to characterize whether (xg^yg) is the unique minimizer to problem (DP). 

The authors in 0 also aim to estimate the probabilities of success of problem (BP) and problem (DP) with Gaussian random 
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Fig. 2. The optimality condition for (DP) problem. [Left] problem (DP) succeeds. [Right] problem (DP) fails. 


sensing matrices by the approximate kinematic formula. They derive the probabilit}{^ by using the convex (descent) cones. 
Note that, as shown in Fig.and Fig.[^ the affine balls S{-) are dehned as S{a) = {x : ||a: + a||i < ||a||i}. Also note that 
cone{S{a)) = ,a), where cone(S{aj) is a conical hull of S{a). 

In Sections]^ and III we generalize the demixing problem specified in Eq. ( |I.4| i, set the corresponding optimization problem 
to recover the signals of such generalized demixing model, and explore its necessary and sufficient condition for successful 
demixing. 


II. Motivation and Problem Definition 


The demixing problem we discuss in this paper refers to the extraction of two informative signals from a single observation. 
We consider a more general model for a mixed observation z G M™, which takes the form 


z = Axo + Byo, 


(II. 1) 


where xq G and yg G are the unknown informative signals that we wish to find; the matrices A G and 

B G ^j-g arbitrary linear operators (not necessary m < ni or 71 , 2 ). We assume that all elements appearing in Eq. 

o are known except for xg and yg. The broad applications of the general model in Eq. ( |II. 1[ ) can be found in IJ) (and the 
references therein). 


It should be noted that: (1) if yg in Eq. (II. 11 is set to zero, then the generalized demixing model is degenerated to BP; (2) 


The demixing model in HI is a special case of Eq. (II. 1 1 if A is set to an identity matrix and B is enforced to be an orthogonal 
matrix; (3) our generalized demixing model has more freedom in the sense of dimension than that in ||2| because A and B 
can be arbitrarily selected. Moreover, the two components xg and yg in our generalized model are permitted to have different 
lengths. 

'Nevertheless, the authors still fail to calculate the actual probabilities. In fact, they only derive the bounds of probabilities that involve the calculation of 


statistical dimension. Unfortunately, up to now the statistical dimension still cannot be calculated correctly. 
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III. Main Result 


The ground truths, xq and yo, in Eq. (II.l i are approximated via solving the convex optimization problem defined as follows, 


which we call generalized demixing problem (GDP); 


{ minimize ||a;||i 

(III.l) 

subject to ||y||i < ||?/o||i and z = Ax + By. 

We call problem (GDP) succeeds provided {xo,yo) is the unique optimal solution to GDP Our goal in this paper is to 
characterize when the problem (GDP) succeeds. 


Theorem III.l. The problem (GDP) has a unique minimizer (x^y) to coincide with {xo,yo) if and only if 

/ 

null(A) n S(xo) = {0}, 

^ null(B)nS(yo) = {0}, 

-AS(xo)nBS(yo) = {0}. 

Proof: First, we assume that the problem (GDP) succeeds in having a unique minimizer (x,y) to coincide with (xo,yo)- 
p~| Claim: null(A) n ^(xo) = {0}. 

Given hi G null{A) n S'(a;o), we have Ahi = 0. By letting {x',y') = (xq + hi,yo), it follows that z = Ax' + By' and 
lly^lli < ll?/o|li. which means that the point (x', y') is a feasible point of problem (GDP). On the other hand, since hi G S'(xo), 
we have ||xo + = ||a;^||i < ||a;o||i- By the fact that the problem (GDP) is assumed to have a unique minimizer (xo,yo), 

we conclude that hi = 0. 

[ 2 ] Claim: null(B) n S{yo) = {0}. 

Given /i2 G null{B) n S{yo), we have Bh2 = 0 and ||?/o + ^2|ii < ll2/o||i- By letting {x",y") = {xo,yo + h^), it follows 
that z = Ax" + By" and |jy"||i < ||yo||i- Thus, /12 = 0, otherwise [x",y") (xo,yo) will be another minimizer to problem 


(GDP). 

[T] Claim: —AS(xo) H BS{yo) = {0}. 

Given s G —ylS'(xo) C BS{yo), there exist x G S{xo) and y G Siyo) to satisfy —Ax = By = s, ||xo + a;||i < ||a:o||i. and 
ho +y\\i < llyolli- By letting {x'",y'") = (xq + x,yo + v), h follows that z = Ax'" + By'" and ||y'"||i < ||yol|u which 
mean that the point (x'", y'") is a feasible point of problem (GDP). On the other hand, since \\x'"\\i < lixolli, {x'",y'") is 
also an optimal solution. By the fact that the problem (GDP) is assumed to have a unique minimizer {xo,yo), we conclude 
that {x'",y'") = (xo,yo). and therefore x = 0 r*>i, y = 0 h" 2 , and s = —Ax = Or™. 


Conversely, we suppose the point (xo,yo) satisfies Eq. (III.2i. Let (x*,y*) be a feasible point of problem (GDP), we aim 


to show that either l|a;*||i > Ikolli or (x*,y*) = (xo,yo)- 
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Let hi = X* — xq and h2 = y* — yo- Since {x*,y*) is feasible to problem (GDP), z = Ax* + By* = A{xq + hi) + 
B{yij + /12) = z + Ahi + Bh2, which implies —Ahi = Bh2. If ||a;*||i > ||a^ol|i^ then we are done. So, we may assume 
||a:*||i < ll^colli. which means hi S S'(a;o). Moreover, ||t/*||i < ||t/o||i implies /12 G S{yo)- Then, we get the fact that 
—Ahi = Bh2 € —n BS{yo) = { 0 }, namely, hi G null{A) and /i2 G null{B). Therefore, we have 


hi G null{A) n S{xo) = {0} and /12 G null{B) n S{yo) = {0}, 


which means {x*,y*) = {xo,yo) and we complete the proof. 


□ 


We know that S{xo) and S{yo) are the affine fi-balls of the points xq and yg, respectively. However, Eq. is the 

formula, consisting of null spaces of sensing matrices and affine fi-balls. Indeed we can relax the affine fi-ball to be its conical 
hull such as ,a;o) = cone{S{xo)) and ,yo) = cone{S{yo)), and attain the following result. 


Corollary III.l. The problem (GDP) has a unique minimizer {x,y) that coincides with (a;o,j/o) if und only if 

null{A) n ,a;o) = {0}, 

null{B)n3i{\\-\\^,yo) = 

-Gl^(|Ml,,a:o)ni3^(|M|,,t/o) = {0}. 


(III.3) 


We emphasize again that if xq and yo has the same length, as in the standard problem (Eq. 1.4 1 , then ,xo) and 

^(11’111 I yo) will reside in the same linear space and their intersection can be geometrically visible, as shown in Eig.However, 
since matrices A and B have arbitrary dimensions in our model, their geometrical interaction cannot simply be observed. Thus, 
we argue that the derivation of necessary and sufficient condition via combining all of the cones is significantly different from 
standard problems ID, 0. 


IV. Simulations and Verifications 


We conduct simulations to verify the consistency between Theorem HI. 1 and GDP. 


A. Verification Procedures 


The verification steps for practical sparse signal recovery based on Eq. (III.l 1 are described as follows. 


(1) Construct the vectors xq G and yo G with ki and ^2 nonzero entries, respectively. The locations of the nonzero 
entries are selected at random, such nonzero entry equals ±1 with equal probability. 

(2) Draw two standard normal matrices A G and B G then capture the sample 2 = Axo + ByQ. 

(3) Solve problem (GDP) to obtain an optimal solution (xi,yi). 

(4) Declare successful demixing if |jxi — X 0 II 2 < 10“®. 
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In addition, the verification steps for theoretic recovery based on Theorem |III.l| are described as follows. 

(5) Solve min ||a:o + a:||i subject to Ax = 0 to obtain an optimal point X 2 - 

(6) Solve min Hj/q + y\\i subject to By = 0 to obtain an optimal point ^ 2 - 

(7) Solve min Uxo + x\\i subject to Ax + By = 0 and |j?/o + y\\i < ll2/o||i to obtain a pair of optimal points (is, ya). 


(8) Declare success in Theorem III.l if .^ 2 -norms of X 2 , y 2 , X 3 , and y^ are all smaller than or equal to 10 
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B. Simulation Setting and Results 

In our simulations, let ni and 712 be the signal dimensions for signals xg and y^, respectively. Their sparsities, ki and k 2 , 
ranged from 1 to ni and 1 to 77 , 2 , respectively. 

First, we let ni = n 2 = 100 and ki = k 2 . Under the circumstance, the simulation results for both the demixing problems 
in Eq. (III. 1 1 and Theorem III. 1 are illustrated in Fig. where the x-axis denotes the sparsity k and the y-axis denotes the 
number m of measurements. We can see that the performances of these two seem to be identical and it is pretty easy to notice 
a fact that the smaller k is, the easier for sparse signal recovery to succeed. 




70 80 90 100 


10 20 30 40 50 60 70 80 90 100 


Fig. 3. Phase transitions for demixing problems: [Left] Practical recovery of two sparse vectors based on Eq. jlll.lj and [Right] Theoretic recovery 
based on Theorem jlll.!} . In each figure, the heat map indicates the empirical probability of success (black = 0%; white = 100%). 

Second, we consider ni ^ 772 , where 77i = 100 and 772 = 160. Again ki and ^2 ranged from 1 to 771 and 1 to 772 , 
respectively. By additionally considering varying number of measurements, the visualization of recovery results, unlike Fig. 
1^ will be multidimensional. So, we chose different numbers of measurements with 10 < 777 < 100 in the simulations to ease 
observations. The recovery result at each pair of fci and ^2 for each measurement rate (^) was obtained by averaging from 
100 trials. In sum, the simulation results reveal that, if each optimal solution in Steps (5)-(7) is zero, then the point, xg and 
yg, satisfies Eq. (III.2i, and vice versa. That is to say, we can check if xg and yg satisfy Eq. ( III.2[ ) by solving these three 
optimization problems in Steps (5)-(7). 

C. Proof of Feasibility of Our Verification 

Now we prove that why the above verification is feasible. We say that X 2 , y 2 , £ 3 , and yg obtained from Steps (5)-(7) are 


all zero vectors if and only if Eq. (III.2i in Theorem III.l holds. We will validate this claim in the following. 
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Definition IV.l. Two cones C and D are said to touch if they share a ray but are weakly separable by a hyperplane. 

Fact 1. ^ pp. 258-260] 

Let C and D be closed and convex cones such that both C and D ^ {0}. Then 

f‘{QC touches D} = 0, 


where Q is a random rotation. 


Lemma IV.l. Steps (5)-(8) constitute a complete verification to in Theorem\in.l\ 


Proof: We want to prove that Steps (5)-(8) form a valid verification for Eq. (III.2 1 . First, we assume the point (ccq, yo) satisfies 


Eq. ( [Iini l. 


A1 Claim: X 2 in Step (5) is zero. 


Since X 2 is an optimal solution to the problem in Step (5), we have ||a:o + :r 2 |li < ll^^olli which implies X 2 G i5'(a;o); and 
Ax 2 = 0 which is followed by £2 G null{A). That is £2 G null{A) n S{xo) = {0}, and hence £2 = 0. 

A2 Claim: ^2 in Step ( 6 ) is zero. 


The proof is similar to the one in 


A1 


A3 Claim: (xsjt/a) in Step (7) is zero. 

Since (^3,^3) is an optimal solution to the problem in Step (7), we have \\yo + y3||i < ||t/o|li which means y^ G 5 '(t/o); 
l|a:o + ^alli < ll^^olli which says that £3 G S{xo)', and A£3 + By^ = 0 which implies —^0:3 = By^ G —AS'(a;o) C BS{yo) = 
{ 0 }. Thus, £3 G null{A) and y^ G null{B), then 0:3 G null{A) nS'(a;o) = { 0 }, y^ G null{B) r]S{yo) = { 0 } and come to the 
conclusion that ( 43 ,^ 3 ) = ( 0 , 0 ). 

On the other hand, suppose that the optimal solutions £ 2 , 112 , and {£ 3 , 113 ) corresponding to minimization problems in Steps 
(5), ( 6 ), and (7), respectively, are all zeros. 

Claim: null{A) n S'(a:o) = {0}. 


B1 


Given x* G null{A) H S{xo), we have x* G S{xo), meaning that ||xo + a;*||j^ < ||a;o||]^. Furthermore, we also have 
X* G null{A), which implies that a;* is a feasible point of problem in Step (5). Due to the fact that 


ll^olli = ||a:o +X2\\i < ||a:o + a;||i Mx G null{A), 

we have ||a;o + > ||a:o|li- Thus ||a;o+a:*||^ = ||a:o||]^, which means x* belongs to adjacency boundary face 

d:t{S{xo)) of S{xo) at xq, where adjacency boundary face 5*(-) = d{-) H d{cone{ )) is the intersection of boundary of 
itself and boundary of its conical hull (as shown in Fig.|^. Therefore, null{A) touches ,a:o) or null{A)r]S{xo) = {0}. 















By Fact[^ we may assume that “null{A) touches ^(|| • || i, xq)” never happens. So we conclude that “null{A)r\S (xq) = {0}”. 


S(xo) 



adjacency boundary 
face of S(xo) 


Fig. 4. The adjacency boundary face of S{xo). 


B2 


Claim: null{B) n S{yo) = {0}. 


The proof is similar to the one in 


B1 


B3 


Claim: —AS'(a;o) C BS{yo) = {0}. 


Given s* S — H BS{yo), there exist x* G S{xo) and y* G S{yo) such that s* = —Ax* = By*. Since y* G S{yo), 
we have \\yo + y*\\i < ||t/o|liJ together with the fact that Ax* + By* = 0, the point {x*,y*) is a feasible point of the problem 
is Step (7). 

Since = (0,0) is an optimal solution to problem in Step (7), we have 


l|a;ol!i = Iko Tislli < Iko + a;*lli • 

Moreover, x* G S{xo) means ||a;o < ||a;o|li- Thus, ||a;o + = ||a:o|li. *-e., x* G 9* {S{xo)) and s* G 9* (—^^(aJo)). 

Therefore, 


-AS{xo) n BS{yo) C 54-Gl5(xo)), 


which means —ASi{\\-\\.^,Xq) touches B^{\\-\\.y ^yo) or —C BS{yo) = {0}. Due to Fact we may assume that 
“—^^(11-11;^ ,a:o) touches ,2/o)” never happens. So we conclude that C BS{yo) = {0}”. □ 


V. Future work 


We plan to employ Corollary 111.1 to estimate the probability of success under some assumptions by the approximate 
kinematic formula from 0- 


Theorem V.l. (Approximate kinematic formula) 

Fix a tolerance y G (0,1). Let C and K be convex cones in K", and draw a random orthogonal basis Q G Then 

d{C) + 6(K) < d — arjVd ^ PjC C QK f {0}} < y 
S{C) + S(K) > d + arjs/d => PjC H QK f {0}} > 1 — p, 
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where arj '■= yJSlog{^/rj) and 5 means the statistical dimension. 

Definition V.l. (Statistical dimension) 

Let C C be a closed convex cone. Define the Euclidean projection Hq : > C onto C by 

Tlc{x) := argmin \\y - x\\^. 
vec 

The statistical dimension 5(C) of C is defined as: 

5(C) :=E,[|inc(5)f], 

where g N(0,1) is a standard Gaussian vector. 


For the generalized demixing model proposed in this paper, we suppose A G and B G have independent 

standard normal entries, and let z = Axg + Bi/q. For the compressive sensing demixing, we may assume m < ni, m < n 2 , 

and both A and B have full rank. Then, we can derive; 

/ 

m > 5(^(\\ ■ ||i,a;o)) + 

< TO > < 5 (^( 1 ! • 111,t/o))+ar,2v^> 

TO > 5(A^(\\ ■ ||i,xo)) + 5(B^(\\ ■ ||i,yo)) + an^s/m, 

which implies 

/ 

F{null(A) n ^(11 • ||i,a;o) = {0}} > 1 - ryi, 

< ¥{null(B) n ^(11 • ||i,?/o) = {0}} > 1 - P 2 , 

P{-A^(|| • ||i,a;o) n B9(\\ ■ |li,yo) = {0}} > 1 - Pa- 

.. 

On the other hand, we also have 

/ 

TO < 5(S!(\\ ■ ||i,a;o)) - 

^ TO < ( 5 (^(|| • ||l,t/o)) - ari2V^> 

TO < 5(A^(\\ ■ ||i,a;o)) +^(B^(|| • ||i,yo)) - 

which implies 


F{null(A) n ^(11 • |li,a;o) = {0}} < pi, 

F{null(B) n ^(11 • ||i,t/o) = {0}} < P 2 , 

P{-24^(|| • ||i,a:o) nS^dl • ||i,yo) = {0}} < m- 
Apparently, if the number to of measurements is large enough, then successful sparse recovery can be achieved. On the 

other hand, failed recovery is possible due to insufficient number of measurements. But if we want to realize the above derived 


results, computation of the statistical dimensions of A.^(|| • ||i,a;o) and B^(\\ ■ ||i,2/o), indicated in Eqs. (V.l i and (V.2i, 


will be an unavoidable difficulty. 
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VI. Conclusion 

Our major contribution in this paper is to derive the necessary and sufficient condition for a successful generalized demixing 
problem. There is an issue worth mentioning, i.e., Amelunxen et al. have evaluated an upper bound and a lower bound of 
the probability of successful recovery for demixing problem (DP). The reason why we did not do that is due to the known 
unavoidable difficulty raised by the generalized model (GDP problem), that is, “How to compute the statistical dimension 
of a descent cone operated by a linear operator?”. We believe that if this open problem can be solved, we will complete 
the generalized demixing problem with Gaussian random measurements. 
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